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Abstract — The secure multiplex coding (SMC) is a technique 
to remove rate loss in the coding for wiretap channels and 
broadcast channels with confidential messages caused by the 
inclusion of random bits into transmitted signals. SMC replaces 
the random bits by other meaningful secret messages, and 
a collection of secret messages serves as the random bits to 
hide the rest of messages. In the previous researches, multiple 
secret messages were assumed to have independent and uniform 
distributions, which is difficult to be ensured in practice. We 
remove this restrictive assumption by a generalization of the 
channel resolvability technique. 

We also give practical construction techniques for SMC by 
using an arbitrary given error-correcting code as an ingredient, 
and channel-universal coding of SMC. By using the same 
principle as the channel-universal SMC, we give coding for the 
broadcast channel with confidential messages universal to both 
channel and source distributions. 

Index Terms — broadcast channel with confidential messages, 
information theoretic security, multiuser information theory 



I. Introduction 

This paper studies a generalization of the channel resolv- 
ability problem HI, universal coding for the broadcast channel 
with confidential messages ||5|, and the secure multiplex 
coding ifTSl . Therefore, we divide the introductory section to 
three subsections. 



A. Generalization of the Channel Resolvability 

For a given channel W with input alphabet X and output 
alphabet J/, and given information source X on X, Han and 
Verdii IS] considered to find a coding / : D — > A" and a 
random variable D such that the distributions of W{f{D)) is 
close to W{X) with respect to the variational distance or the 
normalized divergence, and evaluated the minimum resolution 
of D to make the variational distance or the normalized diver- 
gence asymptotically zero. In their problem formulation, the 
randomness D used to simulate the channel output distribution 
can be chosen. 

In this paper, we shall consider the situation in which we are 
given a channel W, an information source X, and randomness 
D and asked to find coding f : D ^ X such that W(f{D)) 
is as close as possible to W{X) with respect to unnormalized 
divergence. We shall study how close W{f{D)) can be to W{X) 
in Theorems [TOl and [T2l in Section HVl 
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It has been observed that the channel resolvability coding 
can be used to construct wiretap channel coding ||2l, El, 
lilOj . By using our generalized channel resolvability coding in 
Theorems [TO] and [121 one can construct coding for a wiretap 
channel that can ensure the secrecy of message against the 
eavesdropper Eve when the randomness used by the encoder is 
non-uniform and statistically dependent on the secret message 
that has to be kept secret from Eve. However, one can 
realize much more than the above, as introduced in the next 
subsection. 

B. Secure Multiplex Coding 

In the problem of wiretap coding, there are the legitimate 
sender called Alice, the legitimate receiver called Bob, and 
the eavesdropper Eve. There is also a noisy broadcast channel 
from Alice to Bob and Eve. Alice wants to send secret 
messages reliably to Bob and secretly from Eve. This problem 
was first formulated by Wyner ||2TI . Csiszar and Korner 
generalized Wyner's original problem to include a common 
message from Alice to both Bob and Eve, and determined the 
optimal rate tuples of information rates of the secret message 
and the common message, and the equivocation rate of the 
secret message to Eve, which is measured by the conditional 
entropy of the secret message given Eve's received signal 13]. 
They called their generalized problem as the broadcast channel 
with confidential messages, hereafter abbreviated as BCC. 

The secrecy of messages over the wiretap channel and 
the BCC is realized by including meaningless randomness 
into Alice's transmitted signal. This decreases the information 
rate. In order to get rid of this information loss, Yamamoto 
et al. ifTSl proposed the secure multiplex coding, hereafter 
abbreviated as SMC, as a generalization of the wiretap channel 
coding. In SMC, Alice has T distinct secret messages Si, . . . , 
^T-. The T - I secret messages ^i, S i-i, Si+i, St 
serve as meaningless randomness to hide 5,, for each / = 1, 
T, and Eve can know little about S,, for i - I, . . . , T . 
Yamamoto et al. did not evaluated the information leakage 
of multiple messages 5 ,, , . . . , S ,„ to Eve, and the authors 
analyzed such information leakage in lfT9l . The authors also 
generalized coding in iflSl so that Alice's encoder can support 
the common message to both Bob and Eve. 

There was a restrictive assumption in both llT9l . ifTSll . 
The multiple secret messages Si, St were assumed to 
have statistically independent and uniform distribution. Such 
assumption is difficult to be ensured in practice, and the 
assumption was always questioned at the author's presentation 
in international conferences. The main contribution of this 
paper is to remove that assumption by using the generalized 
channel resolvability coding given in Theorems [TO] and [T2] 
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Specifically, we shall give two code constructions for SMC. 
The first one in Section IV-BI is based on the channel resolv- 
ability coding in Theorem [12] while the second one is based 
on Theorem \TU\ in Section IV-CI The second construction is 
insufficient to prove the capacity region, but can provide a 
better exponential decreasing rate of the mutual information of 
secret messages to Eve under certain situations. By using these 
constructions, we shall evaluate the decoding error probability 
and the mutual information to Eve in Section |V] in the non- 
asymptotic setting. In Section [Vl] we formulate the capacity 
region of SMC, analyze the asymptotic performance of two 
constructions, and prove that the first construction achieves 
the capacity region of SMC. In Section IVIII we shall prove 
that the mutual information to Eve converges to zero when the 
normalized mutual information to Eve converges to zero. This 
property is what is called the strong security ll20l . 

In Theorem [T6l of Section IVl we show that we can have an 
upper bound of mutual information between multiple secret 
messages and Eve's received signal, by attaching random 
group homomorphisms satisfying Condition [TT] to any error- 
correcting code for channels with single sender and single 
receiver or the broadcast channel with degraded message sets 
ifTS I. However, the upper bound in Theorem [16] becomes 
difficult to be computed when the error-correcting code is not 
given by the standard random coding in information theory. In 
Section IVIIII we shall give two upper bounds on the mutual 
information that can be computed easily in practice when 
the construction of Section |V] is combined with an arbitrary 
given error-correcting code. Section IVIIII is enhancement of 
our earlier proceeding paper |[T2l . 

Universal coding is construction of encoder and decoder 
that do not use the statistical knowledge on the underlying 
information system (usually channel and/or source) In 
Section |IX] we shall give a construction of SMC universal 
to channel. The basic idea in Section |IX] is to combine the 
construction in Section [V] with the universal coding for the 
broadcast channel with degraded messages sets in ifTTI . while 
in Sections [V]4VIII the random coding in ||T6| is used as their 
error-correcting mechanism. Section |IX] is enhancement of our 
earlier proceeding paper |[T3l . 

C. Universal Coding for the Broadcast Channel with Confi- 
dential Messages 

Channel-universal coding for BCC had not been studied 
before |T3], and coding for BCC can be regarded as a special 
case of SMC. In Section |IX] and |[l3l we consider SMC 
universal to channel, but its universality to the source, that is, 
the probability distributions of multiple secret messages, is not 
considered. In Section |X] we give a coding for BCC universal 
to both channel and source. Its channel-universality is realized 
by the same principle as Section |IX] and lITSl . Section 1x1 is 
also enhancement of our earlier proceeding paper |[T3l toward 
another direction different from Section |IX] 

II. Broadcast channels with confidential messages 

Let Alice, Bob, and Eve be as defined in Section |l] X 
denotes the channel input alphabet and J/ (resp. Z,) denotes the 



channel output alphabet to Bob (resp. Eve). We assume that 
X, J/, and Z. are finite unless otherwise stated. We denote the 
conditional probability of the channel to Bob (resp. Eve) by 
Py\x (resp. Pz\x)- The task of broadcast channels with confi- 
dential messages is the following. (1) Alice sends the common 
message E to Bob and Eve. (2) Alice confidentially sends the 
secret message S to Bob. Here, we denote the sets of the 
common messages and the secret messages by £ and S. Our 
protocol is given by Alice's stochastic encoder (fa from Sy.& 
to X, Bob's deterministic decoder (pi, : ^ ^ S x S and Eve's 
deterministic decoder ip^ '■ X ^ &■ The triple if - {(fa,'Pb,'Pe) 
is called a code for broadcast channels with confidential 
messages. Then, the performance is evaluated by the following 
quantities. (1) The sizes of the sets of the common messages 
and the secret messages, i.e, |fi| and \S\. (2) Bob's decoding 
error probability Pt[PY\x,<p] Pr{(5,£) ^ <pb(Y)}[PY\x,<p]- 
(3) Eve's decoding error probability Pe[PYix,(p] := Pt{E + 
(Pe{Z)}[Pz\x,(p]- (4) Eve's uncertainty H{S\Z)[Pz\x,>Pa]- Since 
these quantities are functions of the channel and the code, such 
a dependence is denoted by the symbol [Py\x, in the above 
notation. Instead of H{S\Z)[Pz\x,'Pa], we sometimes treat (5) 
leaked information I(S : Z)[Pz\x,'fa]- 

In fact, we sometimes need to evaluate the error probability 
when S and/or E is fixed. In such a case, we denote it by 
Ph(S = s)[Pyix,'P], Pb(S = = e)[PYix,<p], and PAE = 
e)[PY\x,'p]- 

First, we review the formulation when both of the common 
messages and the secret messages are subject to uniform 
distributions. The set S„ denotes that of the confidential 
message and S„ does that of the common message when 
the block coding of length n is used. We shall define the 
achievability of a rate triple (Ri, R^, Rq). For the notational 
convenience, we fix the base of logarithm, including one used 
in entropy and mutual information, to the base of natural 
logarithm. 

Definition 1: The rate triple (Ri, R^, Rq) is said to be 
achievable for the equivocation criterion if the following con- 
dition holds. The size of the sets of the common and confiden- 
tial messages are |£„| - e"'^" and \S„\ - e"'*'. The common and 
confidential messages are subject to the uniform distributions 
on S„ and £„, respectively. There exists a sequence of the code 
fn{fa,n,fb.iufe.n), 1-6. , AHcc's stochastic CHCodcr ifaji from 
S„xS„ to X", Bob's deterministic decoder i^i „ : J/" S„x&„ 
and Eve's deterministic decoder (fe^„ : X" &n such that 

lim PJP" ,^„] =0 

H{Sn\Z")[P"Va,n] 

Uminf > Re. 

n—>oo fi 

The capacity region with the equivocation criterion of the BCC 
is the closure of the achievable rate triples for the equivocation 
criterion. 

Theorem 2: lO The capacity region with the equivocation 
criterion of the BCC is given by the set of Rq, Ri and Rg such 
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that there exists a Markov chain U ^ V ^ X ^ YZ and 

Ri+Ro < I(V;Y\U) + mm[I{U;Y),IiU;Z)], 
Ro < mm[I{U;Y),I{U;Z)], 
Re < I{V;Y\U)-I{V;Z\U), 
Re < Ri. 

As described in lITSi , U can be regarded as the common mes- 
sage, V the combination of the common and the confidential 
messages, and X the transmitted signal. 

Here, we propose another type of capacity region. 

Definition 3: The rate triple (Ri, Ri, Rq) is said to be 
achievable for the leaked information criterion if the following 
condition holds. The size of the sets of the common and 
confidential messages are |£„| = e"*» and \S„\ = e"'^'. the 
common and confidential messages are subject to the uniform 
distributions on S,, and £„, respectively. There exists a se- 
quence of the code f„{'Pa,n,fb,n,'Pe.n), i-C-, AHcc's stochastic 
encoder i^,, „ from S„ x £„ to X'\ Bob's deterministic decoder 
fb.n ■ ^" S„ X £„ and Eve's deterministic decoder 
feji '■ Z," — » &n such that 

lim Pb{PY\x^ ^f„] = 
lim ,^„] =0 

I{,S„:Z")[P"Va,n] 

lim sup < Ri. 

n^oo n 

The capacity region with the leaked information criterion of 
the BCC is the closure of the achievable rate triples for the 
leaked information criterion. 

The capacity region with the leaked information criterion 
of the BCC is characterized as a corollary of Theorem |2] 

Corollary 4: The capacity region with the leaked informa- 
tion criterion of the BCC is given by the set of Rq, Ri and Ri 
such that there exists a Markov chain U — > V — > X — > YZ and 

Ri+Ro < I(V;Y\U) + mm[IiU;Y),I{U;Z)], 
Ro < min[/(f/;F),/(t/;Z)], 
Ri > [Ri-IiV;Y\U) + IiV-Z\U)]^. 

That is, when Ri +/?o < /(V; Y\U) + mm[IiU; Y),IiU;Z)] and 
< min[/(f/; Y), I{U; Z)], there exists a sequence of the code 
fn{fa,n,ifb.iufe.n), 1-6., AHcc's stochastic cncodcr ifaji from 
S„xS„ to X", Bob's deterministic decoder (pb^„ : J/" — > S„xS„ 
and Eve's deterministic decoder tpe,,, : Z" — > £>ii such that 

limPfe[P" ,^„] =0 

n—><x> ' 

hm PJP" ,^„] =0 

n—><x> I 
I(S,r.Z")[P"<Pa.n] 

lim sup < [Ri - I(V;Y\U) + I(V-Z\U)]+. 

«— >oo n 

Next, we consider the BCC when the common and con- 
fidential messages do not obey the uniform distributions on 
S„ and £„, respectively, i.e., the confidential message S 
might have a correlation with the common messages E. when 



the confidential message S is independent of the common 
messages E, 

I(S : Z)[PYix,iPa] < I(S : Z£)[Py|x,^J 
=7(5 : Z\E)[PYix,'Pa] + I{S : 
^I{S ■.Z\E)[Pr^x,'Pa], 

I(S :Z)[PY\X,fa] 

^H(S ) - His |Z) > H(S \E) - H{S |Z) 
=H(S \E) - (HiS \ZE) + 7(5 : £|Z)) 
=7(5 : Z\E) - I{S : E\Z) 
>I{S : Z\E) - H{E\Z) 
>I(S : Z\E) - H{E\^AZ)). 

When the error probability goes to zero exponentially, Fano 
inequality guarantees that H{E\Z) goes to zero. Hence, 7(5 : Z) 
and 7(5 : Z|7i) have the same asymptotic behavior So, even if 
we replace 7(5 : Z) by 7(5 : Z|7i) in Definition[3] we obtain the 
same capacity region. However, when the confidential message 
5 is not independent of the common messages E, 7(5 : Z) and 
7(5 : Z|7i) have the different asymptotic behavior. Since 

7(5 : Z) = 7(5 : ZE) - 7(5 : E\Z) (1) 
>7(5 : E) - H(E\Z) (2) 
>I(S : E) - H(E\^e(Z)), (3) 

7(5 : Z) is lowerly bounded by 7(5 : E) when the error 
probability goes to zero exponentially. That is, when the 
mutual information 7(5 : E) is finite, it is impossible that 
the mutual information 7(5 : Z) goes to zero because Eve can 
infer the secret message from the common message. Thus, 
it is not suitable to treat the mutual information 7(5 : Z) as 
leaked information. Hence, we adopt the conditional mutual 
information 7(5 : Z|7i) as leaked information. 

Now, we propose the concept of "source-channel universal 
code for BCC". In a realistic setting, we do not know the 
distribution of sources and the form of the channel, precisely. 
The distribution of sources might be non-uniform. In order to 
treat such a case, we have to make a code whose performance 
does not depend on the distribution of sources and the form of 
the channel. The performance of such a code should depend 
only on the correlation between input and outputs. Although 
the transmission rates are characterized by the pair (7?o,7?i), in 
order to make a code achieving the capacity region, we employ 
other two parameters R^ and Rp that satisfy Rq < R^ and 7?o + 
R\ < Rc+Rp- Hence, in the following definition of a universally 
attainable quadruple of exponents and equivocation rate, we 
focus on the set (R^)'* of four real numbers 7?p,7Jc,7Jo,7?i. 

Definition 5: A set of functions (£*, E'', E+, EJ) from 
(R+)4x'W(/\^, J/, Z) to [R+ U{0)]4 is said to be a universally 
attainable quadruple of exponents and equivocation rate for 
the family "WiX, if, Z) if there exist a sufficiently small real 
number 6 > and a sufficiently large integer satisfying 
the following condition for 7?p > 0, 7?c > 0, e > 0, 
{Rq,R\) 6 (R^)^. For a given sequence of confidential message 
sets S„ with cardinality e"*' and the common message sets &„ 
with cardinality e"*°, there exists a sequence of codes <!)„ of 



4 



length n of rates at least {Rq,R\) such that 

PbW\ O,,] < exp(-«[£*(/?p, R,,Ro,RuW)- e]), 
Pe[W", 0„] < exp(-«[£^(/?p, R„Ro,Ri, W) - e]), 

and 

i{S,r.z")[w",^„] 

<max{exp{-n[Ei(Rp,R^,Ro,Ri,W) - e]), 
n[E^_(Rp,R„Ro,RuW) + e]}, 



(4) 
(5) 



(6) 



for any sequence of joint distributions for confidential message 
S,, on S,, and the common message E„ on £„, and the «-th 
memoryless extension of any channel W e "WiX, J/, and 
n > A^. 

In order to describe the exponential decreasing rate, as in 
lim . we introduce the following functions. 



i-p 



<P(p,Pz\l,Pl) 

= log2 Y,PLii)iPz\L{z\{)"^'-P^) 

<P(p,Pziv,Pviu,Pu) 
= log ^ Pu(u) ^ Pv|t/(v|M)(Pziv(z|v)'/"-'") 



(7) 



l-p 



Observe that is essentially Gallager's function £0 |7|. Using 
these quantities, we define 

E''iRp,Rc, Py,v,u) 
:= max mm[-pRp - (p(-p, Py\v, Pv\u, Pu), 

0<p<l 



- p{Rp + Rc) - <f>{-p,PYm,Pv,u)] 



and 



E'iRc,Pz,u)-^ max-pR, 

0<p<l 



■^(-p,Pz\u,Pu) 



E^{R,Pz,v.u) max pR - (f,(p, Pz\Vy Pv\u^ Pu)- 

0<p<l 



(8) 

(9) 
(10) 



Suppose that we are given a broadcast W : X ^ 
and positive real numbers Rp and /?c. We fix a distribution 
Quv on U X 'V, a channel H : — > /Y and the rates 
(Rp,Rc,Ro,Ri) e (R+)^ where the RVs U ,V, X, Y and 
Z are distributed according to Quv, S and We present 
a universally attainable quadruple of exponents and leaked 
information rate in terms of Rp, R^^, Quv and H as 

^E\Rp,R„Ro,RuW) = £''(/?p,7;e, ° H) ■ Quv), (11) 
^E''iRp,R„Ro,Ri,W) = £''(7;e,(W o H) • Q^jy), (12) 

£^ =<(i?p,7;c,/Jo,/?i, W) - E\Rp-RuiWoE) ■ Quv), 

(13) 

£f =E[iRp, R^,Ro,Ri,W)^ /(V; Z|?7) - /?p + T^j . (14) 

Theorem 6 (Extension of M7\ Theorem 1, part (a)]): Eqs. 
(fTn i-(fT4li are source-channel universally attainable rates of 
exponents and equivocation rate in the sense of Definition |5] 

The proof will be given in Section |X] after the proof of 
universal code for SMC. 

Remark 7: Csiszar and Korner||5l treated BCC with non- 
uniform information source. However, their formulation is 



different from our formulation in the following point. In their 
formulation, they fixed a correlated non-uniform information 
source on .S x £ and assume that the information source S„ 
and E„ obey its «-fold independent and identical distribution. 
However, in our formulation, we do not assume the indepen- 
dent and identical distribution condition for the distribution 
of the information source S„ and E„. Further, our universal 
code does not depend on the distribution of sources. Indeed, 
information source is not given as an independent and identical 
distribution or known, in general. So, our setting is needed for 
such a case. 

III. Broadcast channels with degraded message sets 

If we set Re = in the BCC, the secrecy requirement is 
removed from BCC, and the coding problem is equivalent to 
the broadcast channel with degraded message sets (abbreviated 
as BCD) considered by Korner and Marton lfT6l . 

Corollary 8: The capacity region of the BCD is given by 
the pair of the rate of common message R^ and the rate of 
private message Rp such that there exists a Markov chain U 
V^X^YZ and 

R^ < mm[I{U;Y),IiU;Z)], 
Ro+Rp < I{V;Y\U) + min[I{U;Y),I{U-Z)]. 

Now, we give a code that attains the rate pair satisfies 

R^ < min[/(f/; Y), I{U; Z)], Rp < I{V; Y\U). (15) 

For a given Markov chain f/ — > V — » X — > YZ, we construct 
an ensemble of codes by the following random coding. For any 
element e Sc, ^dsc) is the random variable taking values 
in 1/, is subject to the distribution Pu, and is independent of 
Oc(i^) with s'e + e Sc- For any element Sp e Sp, Op(ic, ^p) 
is the random variable taking values in 'V, and is subject to the 
distribution Pv\u=<s>(\:), and is independent of ^p{s'^, s'^ with 
(i^, ip) + (ic, ■Sp)- Bob's decoder (D^, and Eve's decoder <bg are 
defined as the maximum likelihood decoders. The quadruple 
(Oc, <I>;,,<l>i,Of) is simplified to 

Lemma 9: iT4l Section IV.B] The above ensemble of codes 
O satisfies the following inequalities. 

E^Pb[PYlv, <1>] <|^p|Pe^(-P'^fl''.^>'|f;,/'f,) 

+ {\S^\\Sp\ye'^^-P-'''<"-'"''"-'''> (16) 



E^PAPziv,^] <l>Sc|V 



-p,Pz\u,Pu) 



(17) 



Markov inequality guarantees that there exists a code C) such 
that 

Pb{PY\v,^] <2|^p|Pe'^(-p./'i'ii'./'i'ii/.^'t') + 2i\S^\\Sp\fe'l'^-P''''^"-'''''"'''^ 
PAPz\v, 4>] <2\Sfe'l'^-P'''^^"'''"\ 

That is, there exists sequence of codes O,, with the rate of 
common message Rc and the rate of private message Rp of 
length n such that 

Ph[P"y^y, <D„] <2e«(P'*"+'^(-P'^>'iV'^''i'''^''» 

4. 2e"(P(*i>+*<)+'^("P-Piw.i''^'t/,i')) (jg) 

(19) 



PeiP'z^y, O,,] <2e"(P«'+'^(-P'^^l'"^''» 
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The above values go to zero with the exponents 
E''(Rp,Rc,PY,v.u) and E'{R^,Pz,u)- The condition ^ 
guarantees that both exponents are positive. In particular, 
Korner and Sgarro ifTTl provided the code that universally 
attains the above exponent in the following way. Assume that 
Pu",v" be the uniform distribution on the sets of sequences 
of length n whose joint type is Qu.v- When we generate 
our code by the above random coding according to the 
joint distribution Pu".v», both error probabilities satisfy the 
following inequalities with probability j|. 

PblPl^v^ 0„] < exp(-n[E''(Rp,R,, Qu,v) - 6]), (20) 
PelPz^v' 4>„] < exp(-«[£^(/?e, Qu,v) - S]), (21) 

IV. General channel resolovability 

In the wire-tap channel model, when the dummy message 
obeys the uniform distribution, channel resolovability JS), ifTOl 
works for guaranteeing the security. In this paper, we consider 
the secrecy when the dummy message does not obey the 
uniform distribution. In such a case, the channel resolovability 
does not work properly. Hence, we need a generalization of 
channel resolovability. 

Assume that a channel ^(ylx) from the alphabet X to be 
sent to the output data Y is given. For a fixed distribution p 
for X, we focus on the average output distribution Wp{y) : = 
Yjx pix)W{y\x) and an encoder A from the message A to the 
alphabet X. This purpose is approximating the average output 
distribution Wp{y) :- Yjx p(x)W(y\x) by the output distribution 
with input A(A). In the original channel resolovability, we 
cosider how well the average output distribution Wp can be 
approximated with a restricted cardinality of the message set 
when the message A obeys the uniform distribution. In the 
following, we consider this approximation problem when the 
message A does not obey the uniform distribution, and we 
call this approximation problem general channel resolovability, 
which is essential for secure multiplex coding with a common 
message with dependent and non-uniform messages. 

Now, we apply the random coding to the alphabet A with 
the probability distribution p. For a 6 A(a) is the random 
variable subject to the distribution p on X. For a + a' e 
A(a) is independent of Ma'). Then, A := {Mfi)]aem gives the 
map from ^ to as a i-> A(fl). For a distribution P^ on 
we can define 

^(pieiiP):=iog^e(fl)i+'p(fl)-'' 

a 

xlf{p\W,p) log^ p.,e'*<^l"''""'''> 

X 

//,+p(A):=-ilogyP^(fl)'" 
P V 

H,^p{A\B) - -- log V P\b)P^^\a\b)'^\ 

a,b 

Hi(A) and Hx{A\B) are defined to be H{A) and H{A\B). 
Since p i-> pH\+p{A), pH\^p{A\B) are concave and 0//i+o(^) = 
OHi+o(A|B) = , 

i/i+p(A) > i/,+p'(A) (22) 
Hi^p{A\B) > Hi^p,{A\B) (23) 



for < p < p'. 

Then, we have the following theorem: 

Theorem 10 (General channel resolvability): For 

p e (0, 1], we obtain 

By applying Jensen inequality to the function x i-> e^. 
Theorem [TO] yields 

E^D{Wp.o^-A\Wp) < ilogEAe^^<"'^"--'ll^"> 
P 

<ilog(l+e-^'^'-<'^)e^<^l^-'^>), 
P 

which is non-uniform generalization of IfTOl Lemma 2]. This 
theorem will be used for the proof of Theorem [T7] 

Proof: Since s i-> ^{^s\WpA^/^-\\\Wp) is convex and 
iA(0|Wp.oA-.||W;,) = 0, 

sD{WpA,^-A\Wp) < ,p{s\WpA,^-A\Wp). 

Thus, 

E^,Mw,.^,-m,) ^ EA2(2i^(fl)WA(,)(y))'^''W7(j) 

y a 

y a a' 

y ^ 

+ 2 P^{a')WMa'){y)rWp''(y)) 

y *3 

+ Ea,a(„) P^ia')WMa'){y)rW-pP{y)) (24) 

a' +a 

+ ^ P^ia'WpWW^^) 

a' +a 

- Ya Xi(EA(«)^^(«)W^A(«)(}')(^(fl)^Aw(3') 

V a 

+ Wp(y)rWp'(y)) (25) 

- Z Z ^<^(a)P^(aWMa)(y)(P^(arW^ia)(yr + WpiyfWp'iy) 

V a 

(26) 

" Z Z ^Ma}P\a)WA(,a)(y)a + i^(fl)'^ WA(«)(y)^W7(y)) 

V a 

"1 + Z Z ^Ma)P\a)'^'WAia){y)'^''WpP(y)) 

= 1 + P^iay^P 2 P.^WAyy^^Wp^iy)) 

a V .V 

= l+(^i^(fl)'+Oe^W»'.rt. 

a 

In the above derivation, (l24l i follows from the concavity of 
X xP, ^ follows from Za'i^a P'^(a') < I, ^ follows 
from the inequality (x + yY < xP +yP. ■ 
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Next, in order to reduce the complexity, we consider the case 
when X and y{ are AbeHan groups. We focus on the following 
condition for the ensemble for injective homomorphisms F 
from J?l to X. 

Condition 11: Any elements x + ^ ^ X and a € the 
relation F{cl) - x holds with probability at most j^px- 

When X and y{ are vector spaces of a finite field F^^, the set of 
all injective morphisms from X satisfies Condition [TT] 

We choose another random variable G ^ X obeys the 
uniform distribution on X that is independent of the choice 
of /x. Then, we define a map Af c{a) '■- F{a) + G and have 
the following theorem: 



Theorem 12 (Affine channel re solvability): Under 
above choice, we obtain 



the 



<1 + g-pHl+p(A|G)glA(P|W,/'mix) 



(27) 



This theorem will be used for the proof of Lemma [ 
is essential for the proof of Theorem [T6] 



which 



Proof: We introduce the random variable := Kf cia) - 
F{a) + G. The random variable Zg is independent of the choice 
of F. For a' e ^, Ap.cia') = F{a' - a) + Z„. Since (|A'| - 
l)Ef|z„WA,,,(«) = (1^1 - l)EfH'f(,,_,)+z„ < S.W, = |^|Wf„,„ 
for a G JH, we obtain Ef\z„WApo(a) ^ pvpV^^'mix a e Jl. 
Further, since x i-> x^ is concave and 2fl-P'^(fl)^ > ^ > j;^, 
we obtain 



and Pmix- Then, we obtain 

y a 

+ Ef|z„ 2 P^fl')H^A..o(«')(3'))''W'pl(3')) 

y a 
' ' a' ta 

_V 

1 - P\a) „ „ X 

< 2 J](Ez„i^(fl)Wz„(3')(^(«rWz„0')'' 

y a 

= J]J](E,/X,<rt»',(.v)((;^)' 

y /II 

+ p\arwzSyrwp'\p))) 

= yp^(«)(i-^r 
4^'^^''^^i-iMi^ 

+ ^ ^ Ez.,i^(fl)'"'^H^z»'"^H^plO')) 

\ a 

+ ^(«)'"' Z Z p>-^.v(3')'"''w^;iO')) 

y .V 



(29) 



(30) 



In the above derivation, dZSl ) follows from Za'^a f''*(fl') < 1, 
( |29b follows from the inequality {x + y)'' < x^ + yP. The final 
inequality follows from ( 128] ). ■ 
In the following, we assume that the input probability space 
X is an Abehan group, and an action of X to the output 
probability space J/ is given. When a channel W from X to 
if is regular in the sense of Delsarte-Piret ||6l, if there is a 
probability distribution Py such that 



)^'<( 



i-i/l^l 



)'^ = 1. 

(28) 



Our proof of Theorem [TO] can be applied to our proof of 
Theorem [H] by replacing A(fl), A| A(a), and P^ by Z„, F|Z„ 



WAy) = f y(x ■ y). 
Since a regular channel W satisfies 

for any G,G' e A", we obtain the following corollary. That 
is, this corollary implies that we need the additional random 
variable G is not needed in the regular case. 

Corollary 13: When the channel W is a regular channel 
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Fig. 1. Communication structure used in Sections FvllIXI 



given by a distribution P on J/, we obtain 

^ pD(Wp^^^-, \\Wp ) 

<1 + e-P^i+M'ii2)e(*(plw.^'mix)_ (31) 

for any G 6 /Y. 

V. Secure multiplex coding with a common message: 

NON- ASYMPTOTIC SETTING 

In the following, we treat secure multiplex coding with a 
common message. In this section, we give the formulation 
of secure multiplex coding with a common message. After 
the formulation, we give two kinds of random construction of 
codes for secure multiplex coding with a common message 
and the evaluation of their performance in the non-asymptotic 
setting. 

A. Formulation and preparation 

In secure multiplex coding with a common message, Alice 
sends the common message to Bob and Eve, and T secret 
messages 5 1 , . . . , 5 r to Bob. We do not necessarily assume the 



uniformity for the distributions of messages S o,S i, . . . ,S j- 
Hence, there might exist statistical correlations among mes- 
sages S o,S I, . . . ,S T- In this scenario, Alice and Bob can 
use S I, . . . ,S Si+i,...,ST for random bits making S, 
ambiguous to Eve. When we focus on 5 7 := {Si : i e 
I) for I c the remaining information S r is 

used for random bits making Sj ambiguous to Eve. These 
messages are assumed to belongs to the sets So,Si, . . . ,St- 
Our protocol is given by Alice's stochastic encoder (fa from 
S X S to X, Bob's deterministic decoder tpi, . ^ S x S 
and Eve's deterministic decoder ipe : X ^ &■ The triple 
(f - {ifa,ipb,fe) is called a code for broadcast channels 
with confidential messages. Then, the performance is eval- 
uated by the following quantities. (1) The sizes of the sets 
of the common messages and all of the secret messages, 
i.e, |>So|, |iSi|, . . . , liSrl. (2) Bob's decoding error probability 
Pb[PY\x,<f] := Pr{(5o,5i,...,5r) * ^fc(F)}[Py|x, (3) 
Eve's decoding error probability PelPYix,^] '■- Pi{So + 
iPe{Z)}[Pz\x,(p]- (4) leaked information I(S i : Z\S o)[Pz\x,'Pa]- 
Instead of I(S j : Z\S o)[Pz\x,'Pa], we sometimes treat (5) 
Eve's uncertainty H{S i\Z,So)[Pz\x,'Pa]- However, when we 
treat the universality of our code, leaked information 7(5 j : 
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Z\So)[Pz\x,'Pa] is used for criterion for performance of our 
code. So, we adopt leaked information I(S j : Z\So)[Pz\x,>fa] 
rather than Eve's uncertainty H{S j\Z, S q)[Pz\x, 'Pa]- 

In order to guarantee that the leaked information is small, 
we employ the method of channel resolvability given in 
Section HV] In order to employ this method, we have to use the 
random coding method. In this section, we propose two kinds 
of random construction for our code. In Subsection IV-BI we 
give the first construction, which, attains the capacity region 
defined in Section |VI-B| This construction has two steps. In the 
first step, similar to the BCD encoder, we apply superposition 
random coding. In the second step, as is illustrated in Fig. 
[T] we split the confidential message into the private message 
B2 and a part Bi of the common message encoded by the 
BCD encoder The coding scheme for BCC in i2 uses this 
kind of message splitting. The average leaked information 
under this kind of construction is evaluated by Theorem [T2l 
which is a affine version of channel resolvability. In Subsection 
IV-CI we give the second construction, which, realizes a better 
exponential decreasing rate for leaked information. However, 
it cannot realize the capacity region. The average leaked 
information under this kind of construction is evaluated by 
Theorem (TO] which is a simple generalization of channel 
resolvability. 

Before giving two kinds of random construction of codes for 
secure multiplex coding with a common message, we prepare 
several useful facts. As in 1111 . we introduce the following 
functions, which are complicated version of if/{p\W,Py. 

if'{p,Pz\L,PL) 

= log ^ ^ PL(i)Pz\L(z\£y^'Pz(zr', (32) 

i//{p,Pziv,Pv\u,Pu) 
= log ^ Pu(u) ^ Pv\u(v\u) 

Proposition 14: Q, IfTTI exp{(f)(p, Pz\l, Pl)) is concave with 
respect to Pi with fixed < p < 1 and Pz\l- For fixed < p < 
1, Pl and Pz\l we have 

exp(i/r(p, PziL, Pl)) < exp(<^(p, Pz\l, Pl)). (34) 
exp(0(p, Pz\L, Pl)) < Ci exp(</.(p, Pz\l, Pl)) (35) 

when Pl < CiPl- 

For a fixed code <1>, any distribution Pz on Z satisfies 

pI{Si:Z\So)[Pz\v,^] 
=pYjPs„(so)IiSi : Z\So = sq)[Pziv,^] 

So 

=P X ^'5o('^o) ^ Ps r\So{sl\so)D(Pz\s j=sj,S„=so,<S>\\Pz\So=-:o,<S>) 
So Si 

< Yj Pso(^o) 2 Psj\So(sT\so)pD{Pzis,-=..i,s,=so,<s>\\Pz) (36) 

So Sj 

and 

pD{Pz\Sj=sr.S,=s,.<^\\Pz) < Hp\Pz\s,=..j,So=so,<s>\\Pz)- (37) 



Thus, Jensen inequality yields that 

^pI{Si:Z\So)[Pziv.'S>] < gE,,„ Psoiso)!,,^ Psj\So{siMpD(Pz,Sj=,j.So=,o,t\\Pz) 

< ^ Ps,(so) Yj PsAS„{si\so)e''P^'''''^--'^-'<>-»-^^^^'^ (38) 

Jo .!/ 

< Y Pso(so) Y ^s,iSo(^/l^o)e^<^l''^i^^-^-^o-»-*"^^'. (39) 

Jo Jj 



B. First construction 

First, we give the first kind of random coding for SMC as 
follows. 

Construction 1: For a given Markov chain {/ — > V ^ 
X YZ, we introduce two random variables Bi and B2 that 
take values in modules Si and S2 and are subject to the 
uniform distribution. We give the following random coding 
by assuming that Si, . . .,St have a module structure. For any 
element (io,^i) e >So x Si, we define the random variable 
•l*e('So,^i) 6 1/ that is subject to the distribution Pu and is 
independent of Oc(s'g,b[) with (s'f^,b\) + {sQ,h\) e xSi. 
For any element (Oc(io, ^1), ^2) x S2, we define another 
random variable <I>p(<l)c('So> ^i)> ^2) £ ^ that is subject to 
the distribution Pv\u=<:>c(so,b\) ™d is conditionally independent 
of <S>p(<^c(sQ,bi),b'2) with ^ b2 given <^c(sQ,bi). Next, 
based on the code <l>a - (Of, O^), we choose an ensemble 
for isomorphisms F' from Si x ■ ■ ■ x St to Si x S2 as 
modules satisfying Condition [TT] We choose the random 
variable G' e Si x S2 that obeys the uniform distribution 
on Si XS2 that is independent of the choice of F'. Then, we 
define a map Af> c'(s) '■- F'(s) + G' and have the following 
lemma: Combining the above three codes, we construct the 
code - (Ofl, Af',c') ■ SoxSi x ■ ■ ■ xSj ^ ^ x^. Similar 
to the case of BCD, Bob's decoder <S>i, and Eve's decoder 
are defined as the maximum likelihood decoders. Hence, 
our code is written by the triple (On, O/,, O^). The structure of 
encoder is illustrated in Fig. [T] 

In order to evaluate the averaged performance of the above 
code (0„, Oi, <l>e), we prepare the following lemma. 

Lemma 15: When we fixed a code <!)„ for the BCD part, 
we obtain the following average performance. 

Ef',G'|o„ exp(p/(5 J : Z\So)[Pz\v, (<1>„, Af-.c)]) 
<EF',G'|a.„ J] Pso(so) Y /^s,|So(^/l^o)e^''*''"'^^-'^-^°-'» *''"''"'^°^'°-*°' 

Jo Jj 

<1 + ^ Ps„(so) Y Ps,\So(siUo)e-''"'^^'-'''^''='"''>^'''^ 

So Jj 

. gl/'(P.^'z|B],B2.So=..o.«<,.^«l,«2' (40) 

where Pb,,b^ is the uniform distribution on Si x S2 and 
Pz\So=so,<s>a is the distribution of the channel output of the 
transmitted codeword <^p{<^c{so, Bi), Bo) with (Bi, B2) obeying 
the uniform distribution Further, due to Corollary [T3] 

in the regular channel case, the inequaUties (|40l i hold even 
when G' is a constant. 
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Applying Jensen's inequality for the convex function x i-> e'^ 
and the inequality log( 1 + x) < x, we obtain 



Markov inequality guarantees that there exists a code O 
(Ofl, O/,, Op) such that 



.5j,So = Jo) 



Ef',g'|o„p/(5j : Z|5o)[Pz|v,(<t«,AF',G')] 
< log(l + 2 Ps„(^o) Yj Psr\s.{si\sa)e-'"''*^^'''^''' 

So Si 

. g'l'(P,PziBi ,S2.io = '0-*" '^"l BiJ^ 

So Si 

. gl/'(P.^'ziB,,S2-So = '0-1'"'^«l-«2\ (41) 

Proof: Applying ( |38] | to the case when Pz - Pz\So=so.'S>„' 
we obtain 

Fr-, ^/(5j:Z|5„)[Pz|l',(t„.Af'.G')] 

Jo Si 
So Jj 

(42) 

For a fixed ij, we apply Theorem [12] to the case when ^ is 
iS/r, X is Si X So, G is G' + F'(sj,Q), which is independent 
of F', and F is the map sjc i-> F'(Q, sjc), which satisfies 
Condition [TT] Then, Af^^c'isj, sjc) = F'isj, sjc) + G' = 
F'(0, ip ) + Zsj. Thus, we obtain 

_j, g-pWl+p(Sjt |S/ = Sj,5o=-!o)gl/'(P.-Pz|B,,«2-So.'I'o'^Bl,«2) (43) 

Thus, we obtain (|40] |. ■ 
Using the above lemma, we obtain the following theorem, 

which gives the averaged performance of the above code 

(Ofl, Ofe, Op). Due to this theorem, we will give the capacity 

region in Subsection IVI-BI 

Theorem 16: The above ensemble of codes <S> - 

(Oa, <I>i„ <I>f.) satisfies the following inequalities 

E^^ exp(pI{Si : Z\So)lPz\v, ^a]) 

So Si 

<1 + l^jl'^e^'^^'+p^'^-r' l'^^''^'')+'^(P'-fzi>''-f''l"'-f'') (44) 

and 

E^Pb[PY\v,^] <\S2fe''''-P-''^'''''''''"'''"^ + (l^oll^iye'^^"^'^''"'''"^''"' 

(45) 

(46) 



\P„<P(-p,Pz\u.Pu) 



E^PAPz\v,m <\SQ\Pe 

Further, due to Corollary [13] in the regular channel case, the 
inequalities ( i44] i hold even when the random variable G' fixed. 

Applying Jensen's inequality to the convex function x i-> e^, 
we obtain 

E^^pI(Si:Z\So)[Pz\v,^a\ 

<log(l + |S[|''e"''^'+p('^-r'l'^-f''^»)+^<^-fzii''/'i'|i''^f)) 

^\^/g^\P g-pH\+p(S lAS I ,So)+ipip,Pz\v,Pv\u -Pu) (47) 



exp(p/(57:Z|5o)[Pz|v,<l>„]) 

<2-^+^ inf (1 + |gj|Pg"P^i+p<'5-r''l'5-r,So)+*(p,/'z|v,^'v!t;,^'(;)) 
" 0<p<l 

<2r+3glplog|Si|-pHi+p(5jH5j,So)+<^(p,/'ziv,Pi'|(;,i't;)]+^ (48) 

/(5j:Z|5o)[Pziv,a>«] 

< jjjf ](g_^]Pg-pHup(Sic\Si,So)+^{p,Pz\v,Pv\u,Pu) ^4(j) 

0<p<I p 

PblPyw,^} 

<f-^^ inf (i^iPg'At-P.'Pt-iv.^'vif/./'f/) + |^o|Pe(*(-Ai'.iJ/,/'i7))^ (50) 

0<p<l 
Pe[Pz\V,^] 

<2^+2 inf I.Sore'^'"'''^^""^''*. 

0<p<l 



(51) 



for all I. Taking the logarithm in (|48] |. we obtain 

I{Si:Z\SQ)[Pz\v,^a\ 
<[log|Si| + -ilj{p,Pz\v,Pv\u,Pu) - Hi^p{S lAS i,Sq)\ 



+ {T + 3) 



P 
log 2 



(52) 



Proof of Theorem \16\ 
We show ( i44l i. Using ( l34l i. we obtain 

Eo ^'''''''''ziBiA.So.'I'o'^Bi.B,) 

<Eo„e**^"^^"'"*2'''»'*'"'^''i-''2^ (53) 
=Eo„ Y^Yj PB,Mbub2)Pz\B,.B,.So,fSz\bub2)^y-P 

=EoEo„|o.J](2^;J^^z|y(z|f,(.o>i,^2))^)'-^ 

z b\.b2 

<Eo V(E^„lo, y -^4^fz|v(z|fp(^o,/^i,^2))^)'-^ (54) 
=Eo. Z ^t'lv(v|a>.(*o,^i))^z|v(z|v)^)'-'' 

=Eo, V/V ^ y Pf/|v(v|0,(io>i))Pz|v(z|v)T^)'-^ 

^ i;" l^il . 

< y y Eo,(-^ y Pv\v{v\^c(.sM)Pz\v{z\v)-^^)'-' (55) 

" Z Z Z ^^^"^^^ Z ^f/n'(H")^ziv(^iv)^)'"'' 
A'Bxf y y Pc/(M)(y Pm(y\u~)Pz\y(Avw^)'-'' 

Z It V 

-\(g^\Pgft'(P-Pz\v,Pv\u,Pv) ^ (56) 

where (i53] l. ( f54l i. and dSSl l follow from ( l34l i. the concavity 
of X i-> x'"'^, and the inequality (x + yf^^ < x'"'^ + y'"''. 
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respectively. Summarizing the above discussion, we obtain 

eP^('5j:Z|So)[fz|l','I>„] 
So sj 

(57) 

=Eo, E/r,_G'|0„ ^ Ps o(so) ^ Psi\So{si\sq) 

So sj 

<2^So(^o)2i's,|s„(ijko) 

So Sj 

■ E(p (1 + e"'''^l+p('^-r'l'^J=''^''^0=''o)e'''<P'^Z|Si,i(2,So.«<,.-f'Bi.B2)) (5g) 

So Si 

. (^l ^ g-pHitp(S fclS i=si,So=so}^^^^p^tl>{p.Pz\v.Pv\u,Pu)-^ ^^g-j 
^. g-pHt^p(S ic\S i,S o)^(g^^p g<p{p.Pz\v,PYW,Pu) 

where (|57li, (|58ll, and §9^ follow from dSS]), Lemma [B] and 
(l56T l, respectively. 

Further, ( l46b follows from Gallager bound. (|45] | can be 
shown similar to ( fT6] l. ■ 



C. Second construction 

Now, we introduce the second kind of random coding for 
SMC. 

Construction 2: For a given Markov chain U ^ V ^ X ^ 
YZ, we define the following random coding. For any element 
So e So, Oc(io) is the random variable taking values in 'ZY, 
is subject to the distribution Pu, and is independent of <l>e(iQ) 
with i[) S() e Sq. For any element (ii , . . . , ij) e >Si x ■ ■ - xSt, 
Op(5o, ii, . . . , 57") is the random variable taking values in 'V, 
is subject to the distribution Pviu^tbAso)' '^^'^ independent 
of ^p(s[^, s\,. . ., s'j) with (s'q, s\,. . .,s'j.) + {so,s\,.. sj~). 
Similar to the above construction, Bob's decoder and Eve's 
decoder are defined as the maximum likelihood decoders. 
Hence, our code is written by the quartet (O^. , O^, O/,, Op). 

The averaged performance of the above code is evaluated by 
the following theorem. Indeed, we cannot derive the capacity 
region from the following theorem. For the strong security, we 
focus on the exponential decreasing rate of the conditional mu- 
tual information. As is explained in Section IvTIl the following 
theorem yields a better bound for this exponential decreasing 
rate than Theorem [T6] in a specific case. 

Theorem 17: The above ensemble of codes <1) = 
(Of, (i>p, Oi, Og) satisfies the following inequalities. 



Eoexp(p/(5j :Z|5o)[i'z|v,<l>]) 

<2 _l_ g-pHitp{S T' \S T,So)+il'(p,Pz\v,Pviu,Pu) 
E^Pb[PY\V,<^] 

<^^^^Pg<p{-p.Pnv,Pv\u.Pu) ^ (\So\\S\y e'^''^'^'''^"^-^'^"-^'' 
E^PAPz\v,<I>]<\Sore'^^-P'''^'"-''^\ 



(60) 

(61) 
(62) 



Applying Jensen's inequality to the convex function x i-> e", 
we obtain 

Eop/(5j:Z|5o)[Pz|v,a>] 
< log(l + '^"if '^fj-j 

^^-pHi^p(Sic\Si,So)+ili{p,Pz\v,Pv\u,Pu) ^(^2) 

Markov inequality guarantees that there exists a code O such 
that 

exp(pI(Si:Z\So)[Pz\v,m 

<2^+^(l + g-P^li+piS i-^o)+il'(p.Pziv,Pviu,Pu)'j 

^2'''+igl-pHup(Si<'\Sj.So)+if'(P,Pziv,Pv[u,Pu)h (54) 

pI(ST:Z\So)lPz\v,<b] 

Pb[PY\V,<^] 

<2^'''^\Sfe'^''^'^'^^''^''''^'"'^"^ + 2''''''^\Sof e'^^^^'^^'"'^"^ (66) 

Pe[Pz\V,^] 

<2'^+^\Sofe''''^-P'''^'"'''"\ (67) 
Taking the logarithm in ( |64] |, we obtain 
I(ST:Z\So)[Pz\v,<b] 

<(T + 3)i^ + [-^(p, Pziv, Pv\u, Pu) - //i+p(5 J. 15 J, 5o)]+. 
P P 

(68) 

Proof of Theorem \17\ 
Inequalities ( |6TI ) and ( |62] | can be shown by Lemma |9] Then, 
we show ( l60l l. 

Then, applying ( l39b to the case with Pz - Pz\u=(i>c{so)^ we 
obtain 

E^^/(S.:Z|S„,0) 

<Eo Psoiso) Ps,|So(^jko)e'^'^"'"'^^='^-^°='°-*"''"'"=*'"»'^ 

So Si 

= X ^•5o(''o) X Psi\So(si\Sf)) 

So Si 

■ E<t,,E<t„|<i).e'^*'^'^^'*^°'^-^°='»-*"^^"'=*''**'-' 

<Xi^s„(^o)Xi^s,is„(^jko) 

.So Si 

■ Eo (1 + g"P^i+p('^-f'l'^J"=^-r"^o=^o'g'A(P'-Pz|v,^'v|u=4>t.(so))) 

= 2Pso(^o)X^S.|So(*jI^o) 

.So Sj 
. (2 ^ g-p/fl+p(Sj^|Sj=.Sj,5o=So)gl/'(p,Pz|v,J'v|[;,P[/)-) 

_l_ ^-pHi+p(Sic\Si,So)gil'(p,Pziv,Pv\u,Pu) 

which implies ( l60l l. ■ 
VI. Secure multiplex coding with a common message: 

ASYMPTOTIC performance 

In this section, we treat the asymptotic asymptotic perfor- 
mance for secure multiplex coding with a common message. 
First, we treat what performance can be achieved by using 
Theorem [16] in Subsection IV-BI without any assumption for 
the distribution of sources. In the next step, we define the ca- 
pacity region under the asymptotic uniformity for information 
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sources. In SMC, this restriction for the sources is essential 
for definition of the capacity region. After this definition, we 
concretely give the capacity region. 

A. General case 

In order to treat asymptotic analysis, for a given set of rates 
(^(O^O' introduce other parameters Rp and satisfying 

T 

i?c+/?p = 2/?„ R,>Rq. (69) 

1=0 

In the following, we denote the set of ((7?,)^^Q,7?p,7?c) satisfying 
the above condition by 'Rj. We choose the code <!)„ satisfying 
(|48T l, ( |49] l. ( |50] l, and (jtlj in the length n case with |.S,,„| := e"*' 
for / = 0, l,...,r, := e«(«'-«o), IS2I e"*p. Then, dSOll 
and dSTT i guarantee that 

lim — log , 0„] >£^(7?p, R,, Pyxu), (70) 

lim — log Pjn,,, 0„] >£^(/?e, Pz,f/), . (71) 

When 

/?e < min[I(Y : U), I(Z : U)], Rp < I(Y : V\U), (72) 

the above both exponents are positive, i.e., the both error 
probabilities go to zero exponentially. Further, due to ( |52] |. 
the leaked information for 5 7,,, can be evaluated as 

i/(5j,„:Z''|5o,„)[P^H„ <!>«,„] 

<[[/?c --Ro]+ + -ip{p,Pz\v,Pv\u,Pu) - -Hi^p(Slc„\Sl.n,So,„)\ 
log 2 



the asymptotic uniformity condition. When the sequence of 
the dummy message S ic „ satisfies 



+ (r + 3)- 

np 

Substituting p - a/n and taking the limits « — > 00, we obtain 
lim sup -I{Si,„ : Z„\SoJ[P'^^y, 0„,„] (73) 

<\iR, - Ro) + I(Z :V\U)- Uminf -i/i+„/„(5 j<,„|5 j,„ 5o,„)l 

+ (r + 3)i^ 

a 

Taking the limits a — > 00, we obtain 

lim sup -7(5 7,„ : Z„\S o,„)[Pl^y, O^,,,] (74) 
«— *oo n 

<[{R,-Rq) + I{Z:V\U) 

- lim liminf -i/i+a/„(5j.,„|57,,„5o,„)l . (75) 

So, the asymptotic performance of our code is characterized 
in dznil, dn), and 

B. Capacity region 

In SMC, we use the message 57^ as a dummy message. 
The secrecy of the message 57 depends on the entropy 
of the dummy message 5 7 . Then, we cannot define the 
capacity region without fixing the entropy rate of the dummy 
message S In order to resolve this problem, we introduce 



hm -H{S r \S i^n,SQj,) = V/?„ 



(76) 



it is called asymptotically uniform. In fact, as is shown as 
Folklore source coding theorem|9j, when the original message 
is compressed up to the entropy rate, the condition ( |76] | holds. 
Even if the dummy message 57^ ,, satisfies the condition ( l76b . 
it is not necessarily uniform or independent of 57 ,, or ^o,,,. 

In order to characterize the limit of the asymptotic perfor- 
mance of secure multiplex coding with a common message, 
we give the capacity region in the following way. 

Definition 18: The rate tuple {Rq, R\, Rj) and the 
equivocation rate tuple {Rij \ + I £ {1, T\\ are 
said to be achievable for the secure multiplex coding with 
T secret messages if the following conditions hold. When a 
sequence of joint distribution for all of the /-th secret 5,,„ on 
S\jt with cardinality e"*' and the common message S'o,;, on 
>So,« with cardinality e"*° satisfies ( l76b for / = 1, . . . , T, there 
exist a sequence of codes i^,, = {<fa,n,ifb.n,fe,n), i.e., Alice's 
stochastic encoder ipa,n from >So,« x.Si,„ x ■ ■ ■ xSt,,, to X", Bob's 
deterministic decoder ipi,^n : J/" Sqj, x Sij, x ■ ■ ■ x ST,n and 
Eve's deterministic decoder : X" — > So,,, such that 

limPi[P" ,^„] =0 
limPJP" ,^„] =0 

n^oo I 

limsup/(57,„ : Z"\S onP^x-Va.n] < Rij 

where Sj,„ is the collection of random variables S with / € 
I, and Y" and Z" are the received signal by Bob and Eve, 
respectively, with the transmitted signal f„{S i_„, . . . , 5 7,,,, E„) 
and the channel transition probabilities Py\x, Pz\x- 

The capacity region of the secure multiplex coding is the 
closure of the achievable rate tuples. 

Theorem 19: The capacity region for the secure multiplex 
coding with a common message is given by the set of Rq, Ri, 
Rt and {Rij \(D ^ I c {I, . . . , T}} such that there exist a 
Markov chain U V ^ X ^ YZ and 

Ro < min[I(U;Y),IiU;Z)], 

T 

Y^Ri < /(y;F|f/) + min[/(f/;}'),/(;7;Z)] 

1=0 

R,j > iJ]Ri + I{Z:V\U)-I{Y:V\U)]^ (77) 
for all 9i J c {!,..., r). 

Proof. The converse part of this coding theorem follows from 
that for Theorem |2] with the uniform distribution on the total 
message sets. We have to show the direct part. Choose a 
sufficient small real number e > and (RdJ^o ^'^^ i - 0,1, . . . ,T 
satisfying 



Ra < mm[I(U;Y),I(U;Z)], 



(78) 



T 

iJ^Rd + e < IiV;Y\U) + mm[I(U;Y),I(U;Z)]. (79) 

1=0 
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Assume that the sequence of the joint distribution of infor- 
mation source satisfies (|76T l. Then, it is sufficient to show the 
existence of a sequence of codes {O,,) such that 



We obtain 



(80) 
(81) 



and 



lim sup -7(5 j,„ : Z„|5o,„)[i'z|v, <!>„] 
<[J]Ri + I{Z:V\U)-I(Y:V\U) + eU. (82) 

Based on the choices Rp := I(Y : V\U) - e and : = 
Tj=o^i " we construct the code <!>„ in the above way. 
Since the conditions ( fTSl l and (|79] l guarantee the conditions 
( |72] |. we obtain dSOl l and dSTT i. The condition ( l76b and Lemma 
l26l guarantees 

lim liminf -//i+„/„(5jc,„|5j,„,5o,„) = V 7?,-. (83) 
(I25]l and dgS) yields 



lim sup- 7(5 j,„ : Z„|5o,„)[^'zn/, <!>„,„] 



<[(7?e-7?o) + 7(Z: y|f/)-^7;,]+ 

r 

^[R,-J]Ri + I{Z:V\U) + Y,Rih 

1=0 (£7 

= [-7?p + 7(Z: y|f/) + 27?,]+.. 



(84) 



iel 



Therefore, since Rp = 7(7 : V | f/) - e, (Hill implies 



VII. Secure multiplex coding with a common message: strong 

SECURITY 

In this section, we show the strong security for secure 
multiplex coding with a common message. 
When conditions ( fTSl ) and (|79] | holds and 

Um lim inf -Hi+piS rJS i^„,S q^,,) 



>yRi-I{Y: V\U) + I{Z : V\U), 



i=l 



we can show that the above code satisfies 

lim I(Si,„:Z„\So,„)[P"<:>„] =0. 

(I — »nn I 



(85) 



(86) 



This property is called strong security. Then, we choose Rp to 
be 7(y : V\U) - min[ ^^^ ^^2^ ^" <185]l ^ which guarantees 
dSOl l and dSTT i. Due to ( |49] l, the leaked information for 5 j_„ can 
be evaluated as 

I(ST,n:Z„\So.n)[Pziv^<^"] 



lim — log7(5j,„ : Z„|5o,„) 

1 

>p hm -77i+p(5j<,„|5j.„,5o,«) 

moo n 

- p(7?e - Rq) - 4>(P, Pz\v, Pv\u, Pu)- 



(87) 



Therefore, 



lim -ip lim -77i+p(5 j<_„|5 j.,„ 5 o,„) 

p->0 n \ n->oo ri 



- p{R, - Ro) - <p(p, Pz\v, Pv\u, Pu)) 
= limliminf -77i+p(5j.,„|5j,„,5o,„) - VRi+Rp - /(Z : V\U) 

p^O /!-><» « 

/=1 

> ^ ( Um hm inf - 77i +p(5 jc,„ |5 /,„ , 5 o,„) 
r 

-YjRi + iiY -.vim-iiz-.viu)) 

i=l 

>0. (88) 



Hence, RHS of dSTl l is strictly positive with small p > 0. Thus, 
we obtain (|86] |. 

When we assume a stronger condition: 



lim -Hi+p(S jc JS J „, S o,n) - y^Ri 

n—too n 



(89) 



for p e (0, 1], using dSTl i. we obtain 



lim — log 7(5 j,„ : Z„|5o,„)[PV' I*"] 
>p(7;p - 2 7?,) - 0(p, Pziv, i^vif/, i'c/). 

Hence, we obtain the following theorem. 

Theorem 20: Assume that the condition ( [89] l holds. For 
given {Ri)J^Q, we choose Rp and 7?^ as follows. 



7?c>7?o, Rc+Rp^Yj^'- 



/=() 



Then, there exists a sequence of codes O,, such that 

lim — log Pb[P'^,y, 0„] > E''{Rp, R„ Pyxu), (90) 

lim — logPJP" ,<£„] > E\R„Pz,u)- (91) 

lim — log7(5j,„ : Z„|5o,„)[7^|v, €>„] > £^(7?p - V 7?,, Pz,i/.c/), 

(92) 



where the functions E", and £''^ are given in dSt-dTot. 
When Rp - Yjiei^^i > ^(^ ■ ^lU), the above exponent is 
positive, i.e., the leaked information goes to zero exponentially. 
In particular, when 

T 

Y^Ri < I(Y : V\U), Ro < min[7(f/; F), 7(t/; Z)], (93) 
i=i 
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we can choose Rp and Rq hy Rp :- Yj^i Ri and R^ = Ro- Then, 
the above inequalities can be simplified to 

lim — log Pbinv^ <!>«] > V Ri, Ro, Pyxu), (94) 



i=\ 



lim — logPJP" ,0„] > &{Rq,Pz.u)- 



(95) 



lim — log 7(5 j,„ : Z„|5o,„)[P^|^, 0„] > E^V 7?,-, Pz,i/,c/)- 

(96) 



The exponent ( |96] l can be improved by using Theorem [TTl 
when the condition ( |93l l holds. In this case, we choose the 
code O,, satisfying (|64] |. (l65T l. ( |66] |. and ( l67b in the length « 
case with \SiJ :- e"*' for ; = 0, 1, . . . , T. In the code O,,, the 
exponential decreasing rate of error probabilities are evaluated 
in the same way as (|94] | and (l95T l. Similarly, using ( |68] |. we 
can show that 

lim sup -7(5 j,„ ; Z„\S o,„)[P"ziv'^>^] 
<[7(Z: y|C/)-^7?,]+ 

^[IiZ:V\U)-IiY:V\U) + J]Rih (97) 



iel 



because Hmp^o j;^(p, Pz\v, Pv\u, Pu) - ■ V\U). From 
in the case ( |93] l, we have 



I{Si,„:Z„\So,n)[P"z>v^^n]< e' 



M-P Xiel'- Ri+>ll{p,Pz\V,Pv\V,Pu)) 



Then, 



Um — log7(57,„ : Z„\S o^„)[P"^n\ 

«— *oo Yi I 

>pY^Ri - 4>(p,Pz\v,Pv\u,Pu)- 



Hence, we obtain the following theorem. 

Theorem 21: Under the same condition as Theorem |20] 
there exists a sequence of codes O,, satisfying ( |94] l, ( |95] l, and 

lim Zii0g7(5j,„ : Z„|5o,„)[7^V' ^ E'^iYRi^Pzxu), 

n—>oo n ^1' 

(98) 

where 

E-^(R,Pz.v,u) := maxpR-i^ip, Pz\v, Pv\u^ Pu)- 

0<p< 1 

Due to (O, (l98]l is better than ^ when (|93l). However, (|92ll 
is more general than ( l98T l. 

VIII. Practical Code Construction 



In Section IVIIII we consider how we can construct prac- 
tically usable encoder and decoder for the secure multiplex 
coding. We shall show that how to convert an ordinary error 
correcting code without secrecy consideration to a code for 
the secure multiplex coding. 



A. First Practical Construction 

We consider to attach F' and G' in Lemma [15] to an 
arbitrary given error-correcting code in order to obtain a code 
for secure multiplex coding. To use the code with F' and G' 
attached, we have to evaluate the amount of information leaked 
to Eve. Theorem [16] provides the average amount of leaked 
information averaged over the ensemble of the error correcting 
code Oa = ((tc, <6p) while Lemma[T5]gives the average amount 
for a fixed error correcting code Oa = (<l)c,Op). However, the 
RHS of (|40] | is too complicated to be computable in practice. 
A computable upper bound is given as follows. 

< max YsYj ^v(v)Pz|v(z|v)^)'-^. 
When Pz\v(z\v) is n-fold memoryless extension, by 

max^(^Pv,,(y)Pz"|y"(z|v)^)'"^ = e«0»»(P.^z,v)^ 



where 



,(p,Pz|v) := \ogm?ixY/YjPv{v)Pz\v(z\v)^^)'-P . (99) 



Observe that RHS of ( 199b can be easily computed by the stan- 
dard convex optimization because Y^ziYiv Pv{v)Pz\v{z\v)~y^'' 
is a convex function of Py with fixed p JT)- Then, we obtain 
an upper bound: 

Ef C'l* gPl<-Sl-Z\So)[Pl^y.{<!>„.^F'.a')]) < I g-pHi^p{S ic\S i.So) ^n4,{p,Pzv) 

(100) 

Suppose that we are given arbitrary error-correcting code 
On for the memoryless channel Pyz\v- The code can be, 
for example, an LDPC code or a Turbo code. By using Eq. 
(llOOt . from Of, we can construct a code for secure multiplex 
coding as follows: 

1) For each nonemptyset I Q {1, T], fix maximum 
acceptable information leakage ej for 7(5 j : Z\Sq). 

2) Fix maximum acceptable probability ei for a chosen 
F',G' not making 7(5 j : Z\So) below ej. 

3) By using (II 00b compute information rate for each secret 
message S i so that 

Ef'.c'\<!>J(Si : Z\So)[P%y,{<i^a,AF,,c')] < eieill" 
for every I. 

4) Then the probabiUty of choosing F' and G' making 
7(5/ : Z\Sq) < ej is > 1 - ei. 

Further, the value (f>mia{p,Pz\v) can be calculated in the 
regular case as follows. 

Lemma 22: When the channel Pz\v is regular in the sense 
of Delsarte-Piret |[6l. 



0max(P, Pz\v) - <P{P, Pz\V, T'v.mix), 

where Pvmix is the uniform distribution on 'V. 



(101) 
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Proof: First, we choose P'y such that 

(pmiixiP, Pz\v) = 4){p,Pz\V,P'v)- 

Define for vq e by 
Then, 

The concavity e*^,Pz\v.Pv) implies that 

y J_g0(P.'Pz|V,/"v,„„) < g0(p,/'z|V,Zv„<E'V ri^/"K,.„) ^ 0(p,/'z|l',fv,mix) 

which implies (llOlb . ■ 

B. Second Practical Construction 

Now, we consider the case when there is no common 
message and a code Op : S2 — > is given. In the following, 
we assume that a module structure in V and the map Op is 
a homomorphism. However, we do not necessarily assume 
any algebraic structure in the channel Pz,y\v^ now. Then, we 
modify the random code given in Lemma as follows. We 
choose an ensemble of isomorphisms F' from S\ x ■ ■ ■ xSj 
to S2 satisfying Condition [TT] We choose the random variable 
G" € that obeys the uniform distribution on ^ that 
is independent of the choice of F'. When Op is a linear 
code such as an LDPC code or a Turbo code, G" means 
that we are using the set of codewords having a common 
nonzero syndrome, while the ordinary linear code consists of 
codewords having the zero syndrome. Then, we define a map 
A/r/ := Op o F'{s) + G" and have the following lemma: 

Lemma 23: 

Ef'C'IO eP^('^-r:Z)[FJn„(Op,AF',G")]) < J ^ g-pH,^p{S Te\S j) ^<p(P.Pziv,Pv.^i.) ^ 

Thus, in the «-fold memoryless extension case, we have 

EfC'IO e^'^^i-^^^Pziv^^-^F'-O'^ < 1 +g-pHl+p(5jH5j)g/«*(p,-Pz|l'./'Km,x)^ 

(102) 

Proof: Now, we define the random variable H e 
^/Op(S2) that obeys the uniform distribution. Let {yi,} be the 
set of coset representatives. Let G' be the random variable 
subject to the uniform distribution on Si. Then, G" is given 
as Op(G') + yn- 



:1 + EhIO Ef' 



/f|<tpC'F',G"|//,Or 



p/(Sj:Z)[P" ,(Op,Ap,G")] 



Op we can construct a code for secure multiplex coding (with 
no common message) as follows: 

1) For each nonemptyset I c {1, T], fix maximum 
acceptable information leakage ej for I(S j : Z). 

2) Fix maximum acceptable probability €2 for a chosen 
F',G" not making I{S i : Z) below ej. 

3) By using (1102b compute information rate for each secret 
message S , so that 

Ef',G"|a.p/(5j : Z)[P«n„(Op,A^-,c")] < ^26^/2^ 
for every I. 

4) Then the probability of choosing F' and G" making 
7(5 J : Z\So) < ej is > 1 - £2- 



IX. Universal coding for secure multiplex coding with a 

COMMON MESSAGE 

In order to treat universal coding for the multiplex coding 
with a common message. 

We introduce the universally attainable exponents for the 
multiplex coding with a common message by adjusting the 
original definition for the BCD given by Korner and Sgarro 

El. 

Definition 24: Let ^(X, J/, Z) be the set of all discrete 
memoryless broadcast channels W : X and the 

set of positive real numbers. A set of functions (£'', E'', {E^, 
E^)i<zii.....T]) from "Rt x "WiX, J/, Z) to [R+ U {0}f"'+^ is 
said to be a universally attainable quadruple of exponents and 
equivocation rate for the family 'WiX, J/, Z) if there exist a 
sufficiently small real number 6 > and a sufficiently large 
integer satisfying the following condition for Rp > 0, R^ > 
0, e > 0, (Ri)i=Q.i,...j 6 (R^)^^'. When a sequence of joint 
distribution for all of the ;-th secret S ,.„ on Sij, with cardinality 
e"*' and the common message S o,„ on So,n with cardinality e"*" 
satisfies 

-H(Sj.,,\Sj,„So,„) > ViRi - 6) (103) 
n f-* 

for i - 1, . . . , r, p € (0, 1] and n > N, there exists a sequence 
of codes 0„ of length n of rates at least (7?,),=o,i,...,r such that 

PblW", 0„] < sxp{-n[E\Rp,R,, (Rdi=o,i,...,T, W) - e]), (104) 
Pe[W\ 0„] < exp{-n[E'{Rp,R„ (Rdi=o,i,....T, W) - e]), (105) 



and 



/(5j,„:Z„|5o,„)[r',0„] 
< max{exp(-«[£f(7;p, R„ (/?,),=o,i,...,r , W) - e]), 
n[E^_{Rp,R„(Ri)i=o,i,...,T,W) + £]], 



(106) 



Suppose that we are given arbitrary error-correcting code Op 
for the memoryless channel Py\v- By using Eq. ( 1102b . from 



for the «-th memoryless extension of the channel W e 'WiX, 
J/, Z) and n > N. 

Suppose that we are given a broadcast W : X ^,Z 
and positive real numbers Rp and ^c- We fix a distribution 
Quv ovi H X 'V, a channel H : 'V — » A" and the rates 
{Rp,Rc,{Ri)i=o,\,...j) e Kt, where the RVs U ,V, X, Y and 
Z are distributed according to Quv, S and W. We present 
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a universally attainable quadruple of exponents and leaked 
information rate in terms of Rp, R^^, Quv and H as 

=E\Rp,R„(Ri)i=Q,i.....T,W) 

^E''(Rp,R„(W^E)-Quv), (107) 
E' ^E'(Rp,R„{Rdi=o.i.....T,W) 

=E'iR„iWoE)-Quv), (108) 
El =Ei{Rp,R„{Rdi=o.i.....T,W) 

=£^/?p - 2 Ri, (W o B) ■ Quv), (109) 

E^_ ^E^_(Rp,Rc,{Ri)i=o.i T,W) 

^I(y-Z\U)-Rp + Y^R„ (110) 

which are given in (|70ll, (EB, (|92]i, and ( l84b . 

Theorem 25 (Extension of M 71 Theorem 1, part (a)]): 
Eqs. ( |107t -( fTT0l i are universally attainable rates of exponents 
and equivocation rate in the sense of Definition |24] 
Proof. We shall attach our evaluation to the constant compo- 
sition code used by Korner and Sgarro. We do not evaluate 
the decoding error probability, because that of our code is not 
larger than flTl . Observe that our exponents in Eqs. jlQli and 
dlQSI ) are the same as IfTTl with the channel W o E. We shall 
evaluate the mutual information. 

We assume for a while that Quv is a type of a sequence 
of length « over 1/ x 'V. Recall that their codebook ifTTl 
Appendix] in the random coding is chosen according to 
the uniform distribution on the sequences with joint type 
Quv- We apply the random coding - i<i>c.n,'^p,n) for 
message So^„,Bi „,B2,n and the other random coding A^c.n 
for message S ij,, . . . ,S j.n given in Section [V] That is, the 
encoder is given by Oq,, - {^a,n,^F,c,n)- In the following, 
Bob's decoder <[>i„ and Eve's decoder Op,, are given as the 
maximum likelihood decoder. We treat the ensemble of codes 
<6„ (0„,„,0i,„,0,,„). 

Since we are using the constant composition code as used in 
IfTTl . U" and V" are not i.i.d. RVs. So, the distribution Pv.U" 
satisfies 

Py-|£/»=„(v) <{n+ l)l^^'^le'^|f/(v|M) (111) 
for a fixed u e 1/" by (3" Lemma 2.5, Chapter 1], and 

Pf/»(M)<(« + i)|^ie'^(«), (112) 

by IS Lemma 2.3, Chapter 1]. 

In the following discussion, since we treat the channel W'^ o 

3" : V Z", we simplify it to w"^. Then, ^ yields that 

&x^{<p{p,'W^,Pv\u',Pu-')) 
<{n + l)l«l'l^lexp(0(p,W^,e«|^,e"^)). (113) 

In our code <!)„„, the random variable V" takes values in the 
subset Tn(Qv), which is defined as the set of elements of 'V" 
whose type is Qv- Hence, it is sufficient to treat the channel 
whose input system is the subset T„(Qv) of 'V". Then, we 
have the following convex combination: 

w"z\t,.(Qv)^ 2 '^("^«)"^"' (114) 



where A(Wn) is a positive constant and ^„{Qv) is the family 
of conditional types from 'V" to Z.", which is the V-shell of 
a sequence of type Qv- The joint convexity of the conditional 
relative entropy yields that 

/(5/,„:Z"|5o,„)[W^,0„,„] 

< 2 mn)I{Sl,„:Z"\SQ,n}[W,,^a.n\- (115) 

We can also show that for any element W„ e "WniQv) we 
have 

> ^ PU„(M) ^(^ i^V..|C/..(v|M)(«)W„(z|v))^)'-'' 

tl Z V 

=^(W„)e'**^'**^"'^>"'i''""P''"\ (116) 

Due to (l44l i and the convexity of x i-> e\ we have 

sxpipE^ J{Si,„ : Z"|5o,„)[W„,0„,„]) 
<Ea^,, exp(p/(5j,, : Z"|5o,„)[W,„ 0„,„]) 
<1 +e„,p(W„,PK»,(/»), (117) 

where_f;„,p(W,„Pv",f/») := exp(np(^,. - Rq - XieARi - S)) + 
<p(p,Wn,Pv"\U",Pu"))- Note that W„ is arbitrary and is not 
necessarily memoryless. Taking the logarithm, we have 

Ea,^_/(5/,„ :Z"|5o,„)[W„,a)«,„] 

<-\og{\ +s„,p{Wn,Pv'.u")\ (118) 
P 

Observe that what we have shown is that the averages 
over 0„,^of expipIiSjj, : Z"\So^„)[W„ ^„j ,]) and 7(5 j,„ : 
Z"\SQ,„)[W„,^a,n] are smaller than Eqs. ( fUTb and ( fTTSl l. 

Let p\{n) be a polynomial function of n. We can 
see that with probability of 1 - l/pi(«) the pair Oa,, 
makes exp(pI{S : Z"|5o,„)[W„, O),,,,,]) and 7(5 /,„ : 
•Z"|5'(),«)[W„, 3)a_„] smaller than piin) times Eqs. ( II 171 ) and 
( II 18l l, respectively. Since the inequalities (1104b and ( 11051 ) hold 
at least with probability 1 - ^ = H with random selection of 
0„,„ lini Eq. (24)], one can take pi{n) > ^2^^'\^Vn(Qv)\ 0, 
and by doing so we can see that there exists at least a code Ofl_„ 
such that all elements W„ e "WniQv) satisfies the inequahties 
( fT04l ) and (fTOgj ) and 

I{Si.„:Z"\So.n)lW„,^„.n] 

<^log(l +£, (W,„Pv".c/")) < — e«,p(W„,Pv,,f/«) 
P P 

(119) 

exp(p7(5j,„ :Z"|5o,„)[W„, €>«,„]) 
<pi(n)(l+e„,p(W,„Pv".£/"))- (120) 
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Thus, defining p2(n) := pi(n)(n + l)^'^'^'^^W„(Qv)l we obtain choose an integer N2 such that 



I(Sj,„:Z"\So,„)[Wz,K>] 

< X ^(Wn)IiSl,„:Z"\So,n)[Wn,^a,n] (121) 
W„e'W„{Qv) 



< y i(W„)^e„,p(H'„,Pv",£/") (122) 
H',.e'W„(Gv) 



P 



f^„.p(Wz^Q"v,u) (123) 



W„eW„(Qi') 

Pi(«)IWGv)l(«+i)'^''''^' 



P2(«) 



P 

ei,p(Wz,ev.;/)", 



e«,p(W2,e"y^/) 



(124) 



where ( fHTT i. (fT22] i. and ( fT23] l follow from (fTTsT l, ( fTT9] l, and 
(I113l l and (II 16b . respectively. 

Since \og si^p{Wz,Qv,u) = Rc - Ro - LieAR, - + 
^(p, Wz, Qv\u, Qu), for an arbitrary 5 > 0, we can choose 
a large integer A^i such that 

inf log(^e,,p(Wz,Gv,t/)") 

l/n<p<l p 

< inf log(eipiWz, Qv,u)") + log PiM + log « 

i/«<P<i 

< - niEliRp, R„ (R,)lo, W) - TS) 

for n>N\. Since (1124b holds with any p € [1/n, 1], we obtain 

log/(5j,„ :Z"|5o,„)[Wi,a)„,,] 
< - niEliRp,R„ {Rdlo, W) - T5) (125) 

for n >Ni. 

Since x i-» exp(x) is convex, ( |115b . ( 11161 ). ( |113b and ( |120b 
yield that 

exp(p/(5j,„ : Z"|5o,„)[W^,0,,„]) 

< Yj ^iW„)exp(pI(Si,,:Z"\So,n)[W„,^a,n]) 

W„e'W„{Qv} 

< ^ Pi(n)(l+e„,p(W2,fv«,£/")) 
W„e1V„(ei') 

</7i(«)|'W„(Gv)|(l + e„,p(W^,^v.-,(/")) 
<P2(«)(l+e„,p("W^Z'ev.f/)) (126) 

Taking the logarithm, we have 

I(ST,r.Z"\So,,)[W"2,^a,n] 



Jog(2p2inf) [iog si,p(Wz,Qv,um^ 

< h« . (127) 



Since lim, 



ip^O ■ 



/(5j,„ :Z"|5o,„)[W^,0„,„] 
<RHS of(fT27]) with p = 1/ Vn 
<«(£^(/?,„/?,,(/?,),?:o'^) + 7'^) 



(128) 



for n>N2- 

Therefore, using (11251 ). ( 1128b . we can see that (E'\ E\ El) 
eI) is a universally attainable quadruple of exponents in the 
sense of Definition |24] ■ 

X. Source-channel universal coding for BCC 

We prove Theorem |6] In the following, we treat the same 
ensemble of codes O,, :- (^a,,u^h,n,^e.n) that in the proof 
of Theorem |25] when T - 2, S \ - S , Sq - E and 52 is the 
random number subject to the uniform distribution. 

We prepare formulas similar to dl 131 ): 



exp(<^(p,W:^°Pv"|f/",Pt/«)) 
<(« + D^'l-^l exp(0(p, o QD), 

exp(0(p,"H^,Pv"![/",i'(/")) 
<(n + Dl^l'l-^' exp(0(p, W;, e'^ij,, Ql)), 

<(n + Dl^l'I'^l exp(0(p, W^, Qly)). 



(129) 
(130) 
(131) 



Since we employ a similar construction, it is sufficient to 
treat the channel whose input system is the subset T„{Qy) 
of 'V". Similar to dl 14b . we have the following convex 
combination: 

"w'yWAQv)^ 2 mn,Y)Wn,Y, (132) 

where A{Wnj) is a positive constant and 'WnjiQv) is the 
family of conditional types from to J/", which is the V- 
shell of a sequence of type Qv- Similar to ( II 16b . we can also 
show that for any element W,, € 'WniQv) we have 



,(/>(-p,W'l,,PvJ:iU»,Pu") 

^il'(-p.'^l,,Pu",v") 
'(-p.w"^,°Pv"iui',Pu") 



,lp(-p.W„ Y,Pv»\U" -Pu" ) 



(-p,W„y,Pijnyn) 



.lt>(-p.W„,oPvt\V'< ,Pu<' ) 



(133) 
(134) 
(135) 



Due to (gill and dUll, any W„ 6 ^niQv) and any W„j e 
^n.Y(Qv) satisfy 



npRp +0(-p, W,,.!- ,Pv<\u" ,Pu" ) 
np(Ri,+Ro)+^(-p^„,r,Pu".v) 



Ei{Rp,Rc, {Ri)Up W), we can 



Using d44l) . we obtain 

e s 
■ &Xp{pD{Pz«\s,.=s,E,,=e.Q>J\Pz"\E,,=eAj) 

<1 +e„,p(W„,Pv«,f/»)- 



(136) 
(137) 



(138) 
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Let p3(«) 3\'W„(Qv)\ + |nV„,y(ev)| and p^in) Ip^in). 
Then, from ( fT36] l. ( fT37] ), ( fT38T l, and ( fTTSl ). all of the following 
conditions hold at least with probability of l-p3{n)/p4{n) - j: 

Pe[w„,<:>„] 

2 PeM) 2 ^s„|£„(^k) exp(pZ)(P2.is„=..£„=..o„,„II^Z"|£„=.,4„.„)) 

e S 

<P4in){l+S„^p(W„,Pv;U")), 

I(St,„:Z"\So.„)[W,„^,J) 

<^^log(l +e„,p(W„,Pv,.,£/»)) 
P 

for any W„ e W„(ev) and any W„,y 6 'W„,f(Gv)- 

So, we choose satisfying the all of the above conditions. 
Next, we consider the random variable E„ subject to the uni- 
form distribution on £„. Then, when e is regarded as a random 
variable, the random variable e satisfies all of the following 
conditions at least with probability of 1 - p2{ii)/p4{n) - i: 

Pb(E„ ^ e)[W„j,<:>„] 
P,(£„ =e)[W„,0„] 

Y,Ps.\E„m^W(pD(Pz "|5„=5,£„=e,6„,,jl^Z"lE„=f,a>„,„)) 
s 

<P4{n)Hl+e„,p(W„,Pv..,u")), 

Yj^SAEM\e)D{Pz.,ls,,=s,E„=eAjPz"\E,=e,<i>,J 



distribution on S'^ x £',. Then, 

<p^(^„)3^g«P«P+0(-p.W'„.r,Pi/»|t/" 'Pu" ) 



np(R,,+Ro)+0(-p, W„T./'[;n.l"i ) 



(141) 



I{S„:Z"\E„)[W„,^a,n] 
< 2 Z ^^«(^k)'D(^Z"|5„=.,£„=.,0„,„II^Z"|£„=.,6„,„) 



=/(5„ :Z«|£„ =e)[W„,0«,„] 
,P4(n)^ 



P 



log(l +£„,p(W„,Pv,,f/»)). 



(139) 



Thus, there exist |£„|/2 elements e e £„ satisfies the above 
conditions: So, we denote the set of such elements in We 
choose p3{n) := 2\'W„(Qv)\ + \'W„j{Qv)\- Then, the random 
variable s satisfies all of the following conditions at least with 
probability of 1 - p^(n)l puin) > 

Pb(S„ = i,£„ =e)[W„,y,0„] 

<p^(„-)3^gnpR„+0(-p,Ty,,i.,Pv»lu„ ,P[,n ) ^np(R,,+Ro)+<l>(-p,W„,r,Pu«.v" )-) 

SXp(jjD(Pz,,^s_-,^E,,=e.'hJ\Pz"\E,,=eAj^ 
<P4(n)\l+S„^p{W„,Pv';U"l 
D(Pz" IS „ =s,E„ =e,0„.„ I \Pz" \E„ =f ,*„,„ ) 

<^^li^log(l +g„ (W„,Pv,,f/»)) 
P 

Thus, there exist |>S„|/2 elements s e S,, satisfies the above 
conditions. So, we denote the set of such elements in S'„. 
Now, we assume that S„ and E„ satisfies an arbitrary 



<'-^^ logd + e„,p(W„,Pv«,f/")), 
P 



(142) 



e 



p/(S„:Z"l£„ )[!¥„.*„,„] 



<Y,PESe)J]PsMe)e' 



<P4(nr{l+B„,p{W„,Pv',,U")- 



(143) 



Similar to ( 11241 ) and ( |126l l, defining pi{n) :- p4{ny'(n + 
1)I'W|-|'V||^^^(Q^^)|^ we obtain 

I(S„ : Z"|£„)[W^,0„,„] <El^!^si,p{Wz,Qv,uT (144) 

P 

exp(p/(5„ : Z"\E„)[Wz, ^a,,,]) <P5(n)(l + e„,p(W^, Q^^))- 

(145) 

Further, defining /76(n) := P4(n)\n + l)''^^''''^'|^„(2y)|, we 
obtain 

Pe[W'2,<^n] 

= ^ .}(W„)P,[W„,<1)„] (146) 

W„e'W„(Qv) 

< A{W„)p4{nfe"P'^"^^^-P'^"°''''"'""'^""'>^ (147) 

W„e'W„(Qv) 

< P4(«)'(« + l)""''^'e"'^*°^^^"''''^^°'^'^i-'^"'^ (148) 

W„e'W„(Qv) 

^P4{nfYW„{Qv)\{n + i)i«i^i^ie«p«o+0(-P.r^oe'. » 

-p(^(n)e"^'^°^'^^^P-^^°^''^"'^"^\ (149) 



where ( fT46] l, ( fT47b . and ( fT48] l follow from (fTT4b , ( fT40] i, and 
( fT29b and (fl35b . 

Similarly, using dUB, ([l32l), ([l30ll, ([EB, dES, and (fT34l) . 
we obtain 

Pb[w"y,%A 
<P4{n?YWnAQv)\{n + 1)''"''''^' 

. (g«P«p+'A(-P.»V.Qv|(;'2u) + g«P(«p+«o)+0(-p,W^,Gyy)-) 

_^j(„-)(-g«(pRp+<^(-p.W'j/,Gi'i(;,G(;)) + g«(p(R„+«o)+<^(-p,Wj,,eu.i'))-)^ 

(150) 



Therefore, from ( 11491 ) and ( 11501 ). for an arbitrary 5 > 0, we 
can choose an integer No such that 

\ogPe[W"^,^n\<-n{E\Rp,R„R(,,R,,W)-6), (151) 
\ogPh[w"^,^„]<-n{,E''{Rp,R,,RQ,R,,W)-6). (152) 
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Similar to ( 11251 ), using ( I142l i. for an arbitrary 5 > 0, we can 
choose an integer A^i such that 



\ogI(S„:Z"\E„)[Wz,K,] 
<-n{E^{R,„R,,Ro,RuW)-6) 



(153) 



Appendix 

Lemma 26: Let A„ be a sequence of random variables on 
the set J?[„ with the cardinality e"'* and B„ be a sequence of 
other random variables. 



for n> Ni. 

Similar to ( |128t . using ( |143l l, for an arbitrary 5 > 0, we can jf only if 
choose an integer N2 such that 



iiS,r.z"\E„)[W2,i>„,„] 

<n{E^{Rp,R„Ro,RuW) + 6) 



for n>N2- 

Therefore, using (1151b . (I152l i. ( I153l l, and ( 1154b . we can see 
that E", E+, EJ) is a universally attainable quadruple of 
exponents in the sense of Definition |5] ■ 

XI. Conclusion 

In order to treat secure multiplex coding with dependent and 
non-uniform multiple messages and a common message, we 
have generalized resolvability to the case when input random 
variable is subject to a non-uniform distribution. Two kinds of 
generalization have been given. The first one (TheoremfTOb is a 
simple extension of Han-Verdu's channel resolvability coding 
IS) with the non-uniform inputs. The second one (Theoremll2b 
uses randomly chosen affine mapping satisfying Condition [TT] 
with the non-uniform inputs. 

We have constructed two kinds of codes for the above type 
of SMC. Similar to BCC in ||5|, the first construction has 
two steps. In the first step, similar to the BCD encoder, we 
apply superposition random coding. In the second step, as is 
illustrated in Fig. [1] we split the confidential message into the 
private message B2 and a part B\ of the common message 
encoded by the BCD encoder Employing the second type 
of channel resolvability, we have derived a non-asymptotic 
formula for the average leaked information under this kind of 
code construction. On the other hand, in the second construc- 
tion, the confidential message is simply sent as the private 
message encoded by the BCD encoder. Hence, it has only 
one step. Employing the first type of channel resolvability, we 
have derived a non-asymptotic formula for the average leaked 
information under this kind of code construction. 

Extending the above formula for the first construction to the 
asymptotic case, we have derived the capacity region for SMC 
defined in our general setting, in which, the message is allowed 
to be dependent and non-uniform while it has to satisfy the 
asymptotic uniformity. Using the both formulas. We have also 
derived the exponential decreasing rate of leaked information. 
While the first formula gives an upper bound in any case, the 
second one gives a better upper bound in some specific cases. 

We have also given a practical construction for SMC by 
using an ordinary linear code. 

Further, we have given a universal code for SMC, which 
does not depend on the channel. Extending this result, we 
have derived a source-channel universal code for BCC, which 
does not depend on the channel or the source distribution. 



Um -H{An\B„) = R 



hm -7/i+a/„(A„|B„) = R 

n->oo n 



(155) 



(156) 



(154) for any a > 0. 

Proof: For an arbitrary small number e > 0, we define a 
probability 



K PA-MiaM - - \ogPM\B4a\b) < R - e] 



158b of Lemma |22] guarantees that 



H{A„\B„) < nR - 6n{e- 



1 + ne). 



That is, 



5n < 



R - j;H{A„\B„) 



+ e 



Thus, lim„^co 6„ - 0. ( |159b of Lemma |22] guarantees that 



Hi^ain{A„\B„) > — log((l - 5„)e"(^-«> + 6„) 
a 



Thus, 



Uminf -Hi^,/„(An\B„) > Hminf -- log((l - 5„)e« + 6„) 



^R-e. 



Since e > is arbitrary. 



which implies 



liminf -//i+„/„(A„|B„) > R, 



Um -Hi^,/niAJB„) = R. 

«-»ix> fl 



Combining relation ( |23] l, we obtain the desired argument. ■ 
Lemma 27: Let A be a random variable on the set J[ with 
the cardinality M and B be another random variable. For any 
ei > and 62 > 0, we define the subset of joint distribution 
for A and B 

'P,,,,.M := {PA,B\PA,B{iaM - \ogPA\Bia\b) < logM - ej) < 62)- 

(157) 



Then, 



max H{A\B)[Pa,b\ < log M - e2(e"'' - 1 + ei) (158) 
min H,^p{A\B)[Pa,b\ >-- log((l - 62)^ + ei). 

(159) 

Here, since the region Veu^M is compact, the above maximum 
and minimum exist. 



19 



Proof: For any integer k, we define the set 



References 



^.,..„M.A- := {Pa 



PA[a\ - log PA{a) < log M - ei ) < 62, 
\{a\-\QgPA{a)<\QgM-ei}\^k 



Pe,.e„M := {PA\PA{a\ - logP^Cfl) < log M - ) < 62)- 

The possible range of integer k is the integers in [1, £2Me^'^']. 
Then, 

max HiA)[PA] 
^f(k) := e2(logfc - log e2)(l - e2)(log(M - k) - log(l - €2)). 
Taking the derivative, we have 

/'w - - - 

X M - X 

which is positive when x < Me2- Hence, 
max H{A)[Pa] 

^f{e2Me-") 

=e2(logM - ei) + (1 - e2)(logM + log(l - 626 "^') - log(l - £2)) 
= logM-e2ei +(1 -ei)log(l + 



1 -69 



-) 



<logM - 62^1 + (1 - £1) 



62(1 -g-^') 
1-62 



= log M - e2(e - 1 + ei). 

Since log M-e2(e - 1 + ei) is linear concerning £2, we obtain 
(fT58ll. 



max e 



-PHi^p(A)[Pa] 



^ (1 - £2)— + €1'"' < (1 - 62)— + 62. 



Since (1 - 62)^ + ^2 is linear concerning 62, we obtain 



max e 



-pHi+p(A|B)[P^I«] < /I _ ^f!!l + 
MP 



which implies (|159t . 
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